This relates to integrated circuits and more particularly, to integrated circuits that include circuitry for implementing optical flow computation.
In the context of the present application, optical flow computation refers to a method for detecting motion between two successive video frames. Consider a scenario in which a first video frame records a thrown baseball located at the center of the first video frame and in which a second video frame records the baseball located slightly to the right of the center of the frame. The optical flow algorithm can be used to analyze the differences between the two video frames and to determine that the baseball is moving towards the right edge of the frame (i.e., by computing spatial gradient matrices for each predefined window in successive frames, according to the well-known Lucas Kanade feature tracking algorithm).
Circuits for implementing optical flow on programmable logic devices (PLDs) have been developed. Conventional optical flow implementations on programmable logic devices, however, involve storing a substantial amount of computed information generated during the optical flow algorithm in external memory and arithmetic circuitry. The optical flow performance is therefore limited by the bandwidth of the external memory and the amount of arithmetic circuits available on the integrated circuit. As a result, conventional optical flow implementations can only support a fairly “sparse” flow (i.e., only a small fraction of pixels within a given frame is being analyzed) and a fairly small window size for each gradient matrix computation (i.e., window sizes are limited to no more than 15 by 15 pixels).
It is within this context that the embodiments described herein arise.